Web page transmission, in which a user selects web page content and receives objects, is a core part of the Internet experience for Internet users. While the experience of users is typically a single selection followed by the viewing of a web page that is presented on the screen, the process of presenting the web page on the screen can involve a large number of objects and multiple request/response round trip communications from the user system to a system that is providing the web page.
One method of improving the performance of web page transmission and presentation is HTTP prefetching. HTTP prefetching typically involves pre-requesting content on behalf of a client or browser before a request for that content is actually generated as a typical HTTP request and response in the course of a typical web page transaction. Certain prefetching embodiments involve pre-requesting content based on predictions about a future user selection without any actual action or selection by the user. Other HTTP prefetching systems involve pre-requesting content in response to a user action or selection as part of a web page transaction. In such systems, when content is prefetched it may become possible to satisfy the request for that content locally (with regard to the client or browser) or at a location with a lower latency to the user, thereby negating the need to transmit the request and wait for the response from a content server. For example, in cases where there exists high latency between the client generating the request and the server which responds with the context requested, each negated request/response may avoid the penalty for such latency, thereby potentially reducing the total time required to satisfy the entire series of requests for the client. This may result in an accelerated end user experience.
In some prefetching systems, the system may have a set of metrics for determining when a file should or should not be prefetched. An ideal goal of a prefetcher may be to identify and prefetch all objects relating to a particular requested webpage, and to avoid prefetching objects which are not later requested by a user. For example, when a user requests a web page, the prefetcher may request (e.g., as a proxy for the user) various objects embedded in the webpage in anticipation of those objects being ultimately requested. Under certain circumstances, however, incorrect objects may be prefetched repeatedly based on incorrect models or difficult to quantify exceptions to a rule, resulting in resources being wasted to prefetch an object that will never be used. In certain cases, a prefetcher may miss objects that are embedded in a web page, and it may be difficult to determine which objects associated with a web page will ultimately be requested, or how an object seen at a proxy server relates to other objects. Such circumstances may result in slower performance and increased wait time for a user while a system fetches an object that was missed by a prefetcher.
Additionally, many web browsers function with a system to parse ahead in the container files they download in an attempt to identify objects that will be requested when the web page is rendered. This speculative function by a web browser only involves parsing static files and obtaining candidate child URLs. Fundamentally this approach suffers from the main architectural problem of web sites with respect to performance, which is that they are “nested”, and such systems need one object to construct the another object so that the system is limited in which objects it may operate on and the improvements that may be gained.
HTML5 has a feature called the Application Cache that uses manifest files to attempt to address issue. A web site developer can put a list of all the files that will be needed for the given web page in the manifest file and then put a reference to this manifest file at the top of the top-level html file so that when rendering the page the browser can download first the top level HTML, and then the manifest, and then any files listed in the manifest. This allows for improved parallelism but is limited by the fact that the files in the manifest can only be those found on the same domain as the top-level html. Such a system is also dependent on the web site developer maintaining the list. Lastly, as with simple speculative prefetching, the HTML5 manifest only lists static URLs and contains no provision for prefetching URLs that are dynamically generated via script code executed at runtime.